---
title: "Backend Setup"
description: "Welcome to the Omi backend setup guide! Omi is an innovative, multimodal AI assistant that combines cutting-edge technologies to provide a seamless user experience. This guide will help you set up the backend infrastructure that powers Omi's intelligent capabilities."
---

## Prerequisites 📋

Before you start, make sure you have the following:

- **Google Cloud Project:** You need a Google Cloud project with Firebase enabled. If you've already set up Firebase for the Omi app, you're good to go.
- **API Keys: 🔑**  Obtain API keys for:
    - **OpenAI:** For AI language models ([OpenAI API Keys](https://platform.openai.com/settings/organization/api-keys))
    - **Deepgram:** For speech-to-text ([Deepgram API Keys](https://console.deepgram.com/api-keys))
    - **Redis:** Upstash is recommended ([Upstash Redis Console](https://console.upstash.com/))
    - **Pinecone:** For vector database; use "text-embedding-3-large" model ([Pinecone API Keys](https://app.pinecone.io/organizations/-/projects/-/api-keys))
    - **Modal: [optional]**  For serverless deployment ([Modal Dashboard](https://modal.com/settings#tokens))
    - **Hugging Face:** For voice activity detection ([Hugging Face Access Tokens](https://huggingface.co/settings/tokens))
    - **GitHub:[optional]** For firmware updates ([GitHub Personal Access Tokens](https://github.com/settings/tokens))
    - **Google Maps API Key: [optional]** For location features ([Google Maps API Key](https://console.cloud.google.com/google/maps-apis/credentials))
    - **Typesense Credentials:** For search functionality ([Typesense Cloud Dashboard](https://cloud.typesense.org/clusters))
    - **Stripe Credentials: [optional]** For paid apps payment processing ([Stripe API Keys](https://dashboard.stripe.com/apikeys))

**Note:** If you are not very experienced in backend development, we recommend installing [Homebrew](https://docs.brew.sh/Installation) (for macOS or Linux) or [Chocolatey](https://chocolatey.org/install) (for Windows).

#### Video Walkthrough

<iframe width="560" height="315" src="https://www.youtube.com/embed/JyXQM0B0Gnc?si=_2W2GU5Qr7dn7gC7" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>


## I. Setting Up Google Cloud & Firebase ☁️

1. **Install Google Cloud SDK:**
    - **Mac (using brew):** `brew install google-cloud-sdk`
    - **Nix Envdir:** The SDK is usually pre-installed
    - **Windows (using choco):** `choco install gcloudsdk`

2. **Enable Necessary APIs: 🔧**
    - Go to the [Google Cloud Console](https://console.cloud.google.com/)
    - Select your project
    - Navigate to APIs & Services -> Library
    - Enable the following APIs:
        - [Cloud Resource Manager API](https://console.cloud.google.com/apis/library/cloudresourcemanager.googleapis.com)
        - [Firebase Management API](https://console.cloud.google.com/apis/library/firebase.googleapis.com)

3. **Authenticate with Google Cloud: 🔐**
    - Open your terminal
    - Run the following commands one by one, replacing `<project-id>` with your Google Cloud project ID:
       ```bash
       gcloud auth login
       gcloud config set project <project-id>
       gcloud auth application-default login --project <project-id>
       ```
    - This process generates an `application_default_credentials.json` file in the `~/.config/gcloud` directory. This file is used for automatic authentication with Google Cloud services in Python.
    - Copy the credentials file to your backend directory and rename it:
       ```bash
       # For macOS/Linux:
       cp ~/.config/gcloud/application_default_credentials.json ./google-credentials.json

       # For Windows:
       copy %APPDATA%\gcloud\application_default_credentials.json google-credentials.json
       ```


## II. Setting Up OAuth Authentication 🔐

### Google OAuth Setup

1. **Create OAuth 2.0 Client Credentials:**
    - Go to the [Google Cloud Console](https://console.cloud.google.com/)
    - Navigate to APIs & Services -> Credentials
    - Click **Create Credentials** -> **OAuth 2.0 Client ID**
    - Configure the OAuth consent screen if prompted
    - Select **Web application** as the application type
    - Set the **Name** (e.g., "Omi Backend Auth")
    - Under **Authorized JavaScript origins**, add:
        - `https://your-domain.com` (replace with your actual domain)
        - `https://your-ngrok-domain.ngrok-free.app` (for local development)
    - Under **Authorized redirect URIs**, add:
        - `https://your-domain.com/v1/auth/callback/google` (replace with your actual domain)
        - `https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/google` (for local development)
    - Click **Create**
    - Save the **Client ID** and **Client Secret** - you'll need these for your environment variables

2. **Configure OAuth Consent Screen:**
    - Go to APIs & Services -> OAuth consent screen
    - Fill in the required information about your app
    - Add your domain to **Authorized domains**
    - Add the necessary scopes: `openid`, `email`, `profile`

### Apple OAuth Setup

1. **Prerequisites:**
    - You need a paid Apple Developer account
    - Your app must be registered in the Apple Developer Console

2. **Create App ID:**
    - Go to [Apple Developer Console](https://developer.apple.com/account/resources/identifiers/list)
    - Create a new App ID with **Sign In with Apple** capability enabled
    - Note your App ID (Bundle ID)

3. **Create Services ID:**
    - In Apple Developer Console, create a new **Services ID**
    - This will be your `APPLE_CLIENT_ID`
    - Configure **Sign In with Apple** for this Services ID
    - Add your authorized domains and return URLs:
        - `https://your-domain.com/v1/auth/callback/apple`
        - `https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/apple`

4. **Create Private Key:**
    - Go to **Keys** in Apple Developer Console
    - Create a new key with **Sign In with Apple** enabled
    - Download the `.p8` file and note the **Key ID**
    - You'll need the **Team ID** from your Apple Developer account

5. **Prepare Environment Variables:**
    - `APPLE_CLIENT_ID`: Your Services ID
    - `APPLE_TEAM_ID`: Your Apple Developer Team ID
    - `APPLE_KEY_ID`: The Key ID from step 4
    - `APPLE_PRIVATE_KEY`: The contents of your .p8 file as a string (keep the `-----BEGIN PRIVATE KEY-----` and `-----END PRIVATE KEY-----` lines)

6. **Configure Firebase for Apple Sign-In:**
    - Go to the [Firebase Console](https://console.firebase.google.com/)
    - Select your project
    - Navigate to Authentication -> Sign-in method
    - Click on **Apple** and enable it
    - Add the same configuration values from your Apple Developer setup:
        - **Client ID:** Your Services ID (same as `APPLE_CLIENT_ID`)
        - **Team ID:** Your Apple Developer Team ID (same as `APPLE_TEAM_ID`)
        - **Key ID:** The Key ID from your private key (same as `APPLE_KEY_ID`)
        - **Private Key:** The contents of your .p8 file (same as `APPLE_PRIVATE_KEY`)
    - Click **Save**


## III. Backend Setup 🛠️

1. **Install Python & Dependencies: 🐍**
    - **Mac (using brew):** `brew install python`
    - **Nix Envdir:** Python is pre-installed
    - **Windows (using choco):** `choco install python`
    - **Install pip (if not present):**
        - Follow instructions on [https://pip.pypa.io/en/stable/installation/](https://pip.pypa.io/en/stable/installation/)
    - **Install Git and FFmpeg:**
        - **Mac (using brew):** `brew install git ffmpeg`
        - **Nix Envdir:** Git and FFmpeg are pre-installed
        - **Windows (using choco):** `choco install git.install ffmpeg`
    - **Install opus:**
        - **Mac (using brew):** `brew install opus`
        - **Windows:** You should already have it installed if you are on Windows 10 version 1903 and above
    - **Install PyOgg:**
        - **All Platforms:** `pip install PyOgg`
    - **Install All Required Dependencies:**
        - **All Platforms:** `pip install -r requirements.txt`

2. **Clone the Backend Repository: 📂**
    - Open your terminal and navigate to your desired directory
    - Clone the Omi backend repository:
       ```bash
       git clone https://github.com/BasedHardware/Omi.git
       cd Omi
       cd backend
       ```

3. **Set up Pusher Service: 📡 [Optional]**
    - You don't need to have the Pusher Service running if you do not intend to use the webhooks feature
    - Navigate to the pusher directory:
       ```bash
       cd pusher
       ```
    - Create a copy of the `.env.template` file and rename it to `.env`:
      ```bash
      cp .env.template .env
      ```
    - Set the `SERVICE_ACCOUNT_JSON` environment variable in the `.env` file to the string representation of your Google Cloud service account credentials (`google-credentials.json`). This is used to authenticate with Google Cloud
    - Move back to the backend directory and run the following command to start the Pusher service:
       ```bash
       uvicorn pusher.main:app --reload --env-file .env --port 8000
       ```
    - Optionally you can expose the Pusher endpoint using Ngrok or a similar service

4. **Set up Typesense: 🔎 [Optional]**
    - You don't need to setup Typesense if you do not intend to use the search functionality
    - Create an account on [Typesense](https://typesense.org/)
    - Create a new collection in Typesense with the name `conversations` and use the schema provided in the `typesense/conversations.schema` file
    - Install the Firebase Typesense extension from [here](https://console.firebase.google.com/project/_/extensions/install?ref=typesense/firestore-typesense-search@2.0.0-rc.1)
      - While setting up the extension, use the following values for the configuration:
        - Firestore Collection Path: `users/{userId}/conversations`
        - Firestore Collection Fields: `structured,created_at,discarded,started_at,id,finished_at,geolocation,userId`
      - Create `typesense_sync` collection and add a document named `backfill` with data `{'trigger' : true}` (required only if you already have memories in Firestore and want to sync them to Typesense)
    - Set the `TYPESENSE_HOST`, `TYPESENSE_HOST_PORT` and `TYPESENSE_API_KEY` environment variables in the `.env` file to the host URL and API key provided by Typesense


5. **Set up the Environment File: 📝**
    - Create a copy of the `.env.template` file and rename it to `.env`:
      ```bash
      cp .env.template .env
      ```
    - Open the `.env` file and fill in the following:
        - **OPENAI_API_KEY:** Obtained from your [OpenAI API Settings](https://platform.openai.com/settings/organization/api-keys)
        - **DEEPGRAM_API_KEY:** Obtained from your [Deepgram Console](https://console.deepgram.com/api-keys)
        - **Redis Credentials:**  Host, port, username, and password from your [Upstash Redis Console](https://console.upstash.com/)
        - **Modal API Key:**  Obtained from your [Modal Dashboard](https://modal.com/settings#tokens)
        - **ADMIN_KEY:** Set to a temporary value (e.g., `123`) for local development
        - **ENCRYPTION_SECRET:** The `.env.template` provides a default key suitable for local development (e.g., `omi_ZwB2ZNqB2HHpMK6wStk7sTpavJiPTFg7gXUHnc4tFABPU6pZ2c2DKgehtfgi4RZv`). For production, you must generate a new secure key.
        - **OAuth Authentication Credentials:** From Section II above
            - **GOOGLE_CLIENT_ID:** Your Google OAuth 2.0 Client ID
            - **GOOGLE_CLIENT_SECRET:** Your Google OAuth 2.0 Client Secret
            - **APPLE_CLIENT_ID:** Your Apple Services ID
            - **APPLE_TEAM_ID:** Your Apple Developer Team ID
            - **APPLE_KEY_ID:** Your Apple private key ID
            - **APPLE_PRIVATE_KEY:** Your Apple private key content (as a string with newlines)
        - **HOSTED_PUSHER_API_URL:** Endpoint of your hosted pusher service (if you are using it, see step 3)
        - **Typesense Credentials:** Host, port, and API key from your [Typesense Cloud Dashboard](https://cloud.typesense.org/clusters)
        - **NO_SOCKET_TIMEOUT: (Optional)** Set to `True` to disable the socket timeout for the backend server (socket will stay connected for as long as the app is open)
        - **Other API Keys:** Fill in any other API keys required by your integrations (e.g., [Google Maps API key](https://console.cloud.google.com/google/maps-apis/credentials))

6. **Install Python Dependencies: 📚**
    You have two options for installing the required Python packages:

    **Option A: Using a Virtual Environment (Recommended) 🐍**
    - Use Compatible Python Version (>=3.9,<3.13)
    - It's recommended to use a virtual environment to isolate your project dependencies and avoid conflicts
    - Create a new virtual environment in the backend directory:
      ```bash
      # Create a new virtual environment
      python -m venv venv

      # Activate the virtual environment
      # On Windows:
      venv\Scripts\activate
      # On macOS/Linux:
      source venv/bin/activate
      ```
    - You should see `(venv)` at the beginning of your command prompt, indicating that the virtual environment is active
    - Install dependencies within the virtual environment:
      ```bash
      pip install -r requirements.txt
      ```
    - All packages will be installed isolated from your system's Python installation

    **Option B: Direct Installation**
    - If you prefer not to use a virtual environment, you can install the dependencies directly:
      ```bash
      pip install -r requirements.txt
      ```
    - Note that this approach may lead to conflicts with other Python projects on your system

7. **Code Formatting: 💅**
    - To maintain code quality, we use `black` for formatting with a line length of 120 characters.
    - You must have `black` installed:
      ```bash
      pip install black
      ```
    - To automatically format your code on commit, install the pre-commit hook from the repository root:
      ```bash
      # From the root of the repository
      ln -s -f ../../scripts/pre-commit .git/hooks/pre-commit
      ```

## III. Running the Backend Locally 🏃‍♂️

1. **Set up Ngrok for Tunneling: 🚇**
    - Sign up for a free account on [https://ngrok.com/](https://ngrok.com/) and install Ngrok
    - Follow their instructions to authenticate Ngrok with your account
    - During the onboarding, Ngrok will provide you with a command to create a tunnel to your localhost. Modify the port in the command to `8000` (the default port for the backend). For example:
      ```bash
      ngrok http --domain=example.ngrok-free.app 8000
      ```
    - Run this command in your terminal. Ngrok will provide you with a public URL (like `https://example.ngrok-free.app`) that points to your local backend

2. **Start the Backend Server: 🖥️**
    - In your terminal, run:
      ```bash
      uvicorn main:app --reload --env-file .env
      ```
        - `--reload` automatically restarts the server when code changes are saved, making development easier
        - `--env-file .env` loads environment variables from your `.env` file
        - `--host 0.0.0.0` listens to every interface on your computer so you don't have to set up `ngrok` when developing in your network
        -  `--port 8000` port for backend to listen

3. **Troubleshooting SSL Errors: 🔒**
    - **SSL Errors:** If you encounter SSL certificate errors during model downloads, add this to `utils/stt/vad.py`:
      ```python
      import ssl
      ssl._create_default_https_context = ssl._create_unverified_context
      ```
    - **API Key Issues:** Double-check all API keys in your `.env` file. Ensure there are no trailing spaces
    - **Ngrok Connection:** Ensure your Ngrok tunnel is active and the URL is correctly set in the Omi app
    - **Dependencies:** If you encounter any module not found errors, try reinstalling dependencies:
      ```bash
      pip install -r requirements.txt --upgrade --force-reinstall
      ```

4. **Connect the App to the Backend: 🔗**
    - In your Omi app's environment variables, set the `BASE_API_URL` to the public URL provided by Ngrok (e.g., `https://example.ngrok-free.app/`) (don't forget the trailing / )
    - Make sure your OAuth redirect URIs in Google Cloud Console and Apple Developer Console are updated to use your Ngrok URL for local development

Now, your Omi app should be successfully connected to the locally running backend.

5. **When You're Done: 🛑**
    - If you used a virtual environment, when you're finished working with the backend, deactivate it by running:
      ```bash
      deactivate
      ```
    - This command returns you to your system's global Python environment
    - To reactivate the virtual environment later, just run the activation command again (`source venv/bin/activate` on macOS/Linux or `venv\Scripts\activate` on Windows)

## Environment Variables 🔐

Here's a detailed explanation of each environment variable you need to define in your `.env` file:

- **`HUGGINGFACE_TOKEN`:** Your [Hugging Face Hub API token](https://huggingface.co/settings/tokens), used to download models for speech processing (like voice activity detection)
- **`BUCKET_SPEECH_PROFILES`:**  The name of the Google Cloud Storage bucket where user speech profiles are stored
- **`BUCKET_PLUGIN_LOGOS`:** The name of the Google Cloud Storage bucket where to put logos for uploaded apps
- **`BUCKET_BACKUPS`:** The name of the Google Cloud Storage bucket used for backups (if applicable)
- **`GOOGLE_APPLICATION_CREDENTIALS`:**  The path to your [Google Cloud service account credentials file](https://console.cloud.google.com/iam-admin/serviceaccounts) (`google-credentials.json`). This is generated in step 3 of **I. Setting Up Google Cloud & Firebase**
  - By default, the backend expects to find a file named `google-credentials.json` in the same directory where the application is running
  - If you've followed Option 1 in step 3 of the Google Cloud setup, this will be already set correctly
  - If you prefer to use the default location of the credentials, set this to the full path of your `application_default_credentials.json` file (e.g., `~/.config/gcloud/application_default_credentials.json` on macOS/Linux or `%APPDATA%\gcloud\application_default_credentials.json` on Windows)
- **`PINECONE_API_KEY`:** Your [Pinecone API key](https://app.pinecone.io/organizations/-/projects/-/api-keys), used for vector database operations. Storing Memory Embeddings: Each memory is converted into a numerical representation (embedding). Pinecone efficiently stores these embeddings and allows Omi to quickly find the most relevant memories related to a user's query
- **`PINECONE_INDEX_NAME`:** The name of your Pinecone index where memory embeddings are stored
- **`REDIS_DB_HOST`:** The host address of your Redis instance
- **`REDIS_DB_PORT`:** The port number of your Redis instance
- **`REDIS_DB_PASSWORD`:** The password for your Redis instance (blank by default)
- **`DEEPGRAM_API_KEY`:** Your [Deepgram API key](https://console.deepgram.com/api-keys), used for real-time and pre-recorded audio transcription
- **`ADMIN_KEY`:** A temporary key used for authentication during local development (replace with a more secure method in production)
- **`OPENAI_API_KEY`:** Your [OpenAI API key](https://platform.openai.com/settings/organization/api-keys), used for accessing OpenAI's language models for chat, memory processing, and more
- **`ENCRYPTION_SECRET`:** A secret key used to encrypt sensitive user data, such as conversations, memories, and chat messages. The key must be at least 32 bytes long. The `.env.template` file includes a default key (e.g., `omi_ZwB2ZNqB2HHpMK6wStk7sTpavJiPTFg7gXUHnc4tFABPU6pZ2c2DKgehtfgi4RZv`) which is sufficient for local development. **For a production environment, it is crucial to replace this with your own securely generated random string.**
- **`GITHUB_TOKEN`:** Your [GitHub personal access token](https://github.com/settings/tokens), used to access GitHub's API for retrieving the latest firmware version
- **`WORKFLOW_API_KEY`:** Your custom API key for securing communication with external workflows or integrations
- **`HOSTED_PUSHER_API_URL`:** URL of your omi-pusher service if you're using one. Example: http://omi-push:8081
- **`TYPESENSE_HOST`:** URL of your typesense server. 
- **`TYPESENSE_API_KEY`:** Typesense API key
- **`GOOGLE_CLIENT_ID`:** Your Google OAuth 2.0 Client ID from [Google Cloud Console](https://console.cloud.google.com/apis/credentials), used for Google authentication
- **`GOOGLE_CLIENT_SECRET`:** Your Google OAuth 2.0 Client Secret from [Google Cloud Console](https://console.cloud.google.com/apis/credentials), used for Google authentication
- **`APPLE_CLIENT_ID`:** Your Apple Services ID from [Apple Developer Console](https://developer.apple.com/account/resources/identifiers/list), used for Apple Sign In authentication
- **`APPLE_TEAM_ID`:** Your Apple Developer Team ID, found in your Apple Developer account
- **`APPLE_KEY_ID`:** The Key ID of your Apple private key for Sign In with Apple
- **`APPLE_PRIVATE_KEY`:** The contents of your Apple private key (.p8 file) as a string, including the `-----BEGIN PRIVATE KEY-----` and `-----END PRIVATE KEY-----` lines
- **`BASE_API_URL`:** The base URL of your backend API (e.g., `https://your-domain.com` or `https://your-ngrok-domain.ngrok-free.app` for local development)

Make sure to replace the placeholders (`<api-key>`, `<bucket-name>`, etc.) with your actual values.

## Contributing 🤝

We welcome contributions from the open source community! Whether it's improving documentation, adding new features, or reporting bugs, your input is valuable. Check out
our [Contribution Guide](https://docs.omi.me/developer/Contribution/) for more information.

## Support 🆘

If you're stuck, have questions, or just want to chat about Omi:

- **GitHub Issues: 🐛** For bug reports and feature requests
- **Community Forum: 💬** Join our [community forum](http://discord.omi.me) for discussions and questions
- **Documentation: 📚** Check out our [full documentation](https://docs.omi.me/) for in-depth guides

Happy coding! 💻 If you have any questions or need further assistance, don't hesitate to reach out to our community.
